Extraction of Topical Consumer Products from Weblogs

This paper proposes a new algorithm of associated topic extraction, which detects related topics in a collection of blog entries commenting on a specified topic. The main feature of the algorithm is to evaluate how important a topic is to the collection, according to the popularity of blog entries through Trackbacks and comments. Another feature is to utilize product ontology for excluding unrelated topics. Evaluation results show that the proposed algorithm can capture users’ impressions of associated topics more accurately than TF-IDF. Introduction Consumer Generated Media (CGM) has been one of the major “word-of-mouth” media, and has a significant impact not just on customers’ purchase decision processes but also on companies’ product marketing strategies. Analysis of CGM posed a challenge in product marketing(Facca & Lanzi 2005; Mishne & de Rijke 2006).Sentiment analysis is a typical CGM analysis, which extracts evaluation expressions from CGM contents commenting on a specified product to summarize its reputation. Its technology is still in the early phase of application to practical product marketing. We have developed a reputation analysis system(Kawamura et al. 2007), which retrieves blog entries commenting on a specified product, and then extracts reputation expressions and related products from the blog entries. The system indicates the overall rating of the product reputation (positive vs. negative), and suggests related products that are much discussed in the blog entries. This paper proposes a new algorithm of associated topic extraction for suggestion of related products. The main feature is the evaluation of how important a topic is to a collection of blog entries, according to the popularity of the blog entries, through Trackbacks and comments. Another feature is the utilization of product ontology for topic filtering to extract products relevant to or similar to the given product. The paper also briefly evaluates the proposed algorithm in comparison with TF-IDF, a widely-used topic detection algorithm. Copyright c ⃝ 2008, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. Associated Topic Extraction The purpose of associated topic extraction is to suggest similar or comparable products that are much discussed in retrieved blog entries commenting on a specified product. The straightforward approach to the extraction is to count frequency of occurrence of proper nouns in blog entries through morphological analysis. Morphological analysis segments and tokenizes text strings into a series of meaningful terms, and then adds the terms to complemental information such as part of speech, pronunciation, semantic information. In the straightforward approach, any blog entry and any noun string appearing in blog entries are uniformly enumerated. Although it may appear to be a plausible approach such a uniform approach does not accord with blog readers’ impression. This paper proposes a new algorithm for associated topic extraction, abbreviated to AT. The main feature is that AT finds a blog thread, which is a set of blog entries spanned by Trackbacks, and then biases the frequency occurrence of proper nouns according to the thread size. This is derived from the following belief. Suppose that blog entries conversing about a target product receive many Trackback pings. Then, the entries may be especially interesting to many bloggers, and products mentioned in the entries may be similar to the target product or competitive with it. The degree of association AT (t) for each extracted topic t is formulated as follows. Let TF (t) and IF (t) be the topic occurrence frequency t and the inverse document frequency for topic t. Given a set of blog entries conversing about a certain product, then AT (t) is defined as follows: AT (t) = (TFR(t) + TFIR(t))× IF (t) (1) Note that TFR(t) and TFIR(t) weight a topicality of topic t by correlation between blog entries. Another feature of AT is to utilize product ontology. Product ontology is composed of an is-a relation representing hierarchical categorization of products and an instance-of relation representing mappings of individual products with product categories. Referring to product ontology, AT finds only similar or competitive products mentioned in the blog threads conversing about a target product to exclude out proper nouns other than product names. We have developed product ontology including more than 400 thousand consumer products including cars, DVDs, and home appli-